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On^the face of it,, the biggest ^olic^-releyaiit question about 
compensatory instruction would seysm to be, **I)oes xt/work?**' That i*s, 
does tlie Title' I program, as a whole, help children exposed to it, ,tQ 
learn in school. This question, however, is one which NIE not only did 
not answer, it is^'one whicl\ we chose not to answer. Why did w^ make 
this decision? For three major reasons. 

pirst. Title I funds a variety of services and within a single 
instructional area ds used for progiN^ms widely divergent in focus, 
^content, and dioration. It is not logically possible, therefore, to 
provide a single index of program impact. 

Second, because of these differences, a national eyaljuation of even, 
a single'\^ instructional area poses problems as the data collection 
requireinents are enormous. Implementation of such a study on a 
national level, even if possible conceptually, was not possible practi- 
cally given the cost constraints under which we were working, 

Third, oior job was to help improve Title I. In this part of 
the ^tudy, we took this to mean that we should invest|!gate promising 
approaches. National surveys work best for questions about programs 
which are typically available. The most promdsing programs might or 
might not/^ widely used*- Our quest was not to be able to give Title I 
a report card grade ih response to "Does it work?*' Instead,, it was to 
advarrce systematic- inquiry into the question: **Can it work**? 

We needed, therefore, a different approach, one whi^ would provide 
" information on specific variables which had two characteristics: 
(1) pQlicymakers were concerned about them and (2) policymakers had the 
potential for doing, something about them throttgh tools available at the 



Federal level, such as legislative or regulatory changes. Wijth these 
criteria in, mind, NIE select^ a fairly limited set o£ program 
variables for detailed study — individualized instructional techniques, 
teacher training, instructional setting, and cost. While this particular ^ 
set ^f variables would in all likelihood have also be^n considered impor- 
tant had wi^,. tjaken some other orientation than, one based on policy concerns, . 
a major reason for thiir selection was their importance as policy ^issues. 

Before going on to talk about the policy implications of our frSaings, 
we would likeja say^a little more about these fojrr variables. "And, 
in case there are some doubts abouU why, or even whether, these are in 
fact policy variables, we will summarize briefly why. each was selected. 

Individualized Instruction ; During the debates over the reauthcirization 
of Title I in 1974, Congress heard test?.mony. suggesting that individualized 
instruction, especially, diagnostic and prescriptive techniques, are unusually 
effective. Some consideration was given to mandating -the use of such 
techniques for Title I instruct ion* This was not carried out. Instead, 
further study was called for, with the NIE being directed to examine the 
effectiveness of individualization for compensatory education students 
as a part of its Cojnpens^tory Education Study mandate. 

It/sHould be noted that the definition of individualization included 
in the mandate was- not precise. The exact phrase used was "individualized ^ 
vfritten lesson, plans", only one aspect of what many would call individua- 
lization. In order to understand better ^yhat exactly the Hill folk had ■ 
in mind, we spent some time discussing the problem with Committee staff. * 
In the end we focussed on what some might call "direct" instruction" or 
what we call diagnostic and prescriptive instruction. It had three things 
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^ going for it: (1). CongresKwas interested in it und coild do something 

about it, (2) there was a fair amount df evidence that it had a positive 
impact upon achievement and (3) /it could be examined in a fairly 
strai-ght -forward manner. \ ' ' . 

Teacher Training : The Congressional mandate also specified that' we 
exami^ie issue/s in the area of "teacher training. Again, discussion was ^ - 
needed to clarify Congressional intent. We found that two types of 
teacher training issues were of concern to policymakers. First, were 
Title I funds being used to provide special training for compensatory 
education teachers? (They are, but not to any great extent.) And second, 
does training make a difference? We must say .that we were somewhat hesitaVit 
to get into this second area, as research attempting to link teacher . 
qualifications/experience and -learning outcomes Has produced such incon- 
sistent results. V;e^ finally did include a brief look at this question in the 
Instructional Dimensions Study. We are sorry to say that our analyses 
produced little to enlighten the field. 

Instructional Setting : The emergence of instructional setting as a 
policy variable of interest followed, a quite different course, l^en we were 
developing our overall design for the NIB Compensatory Education Study, -of 
which IDS is a part, we held aeries of meetings with state and district 
' personnel to gather their ins^j^s into the area/issues the NIE study should 
address. An area which repeatedly surfaced was problems caused by the 
practice of pull-out instruction . Many irate teachers and administrators 
, cursed the Title I rules for forcing them to use the pull-out mode. (The 
rules, by the way, do not .include such a requirement.) V/e decided as a 
result of these conversations that an investigation of instructional 

settings,, if aiyd how they relate to achievement, w9uld be important. 
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In our research plans submitted to Congress therefore proposed 

" • ^ ^ •* 

*a look ati the effects of setting. Thi5 proposal was accepted. 

Cost_: The importance of cost as a policy variable probably needs 
' ' ■ ' '' ' . 

little explanation. .Suffice it to say we considered it^ important to have' - 

\ - ' - _ . 

cost data on the practices whose effectiveness we. were examining.' We wanted 

to -be able to sa/, for example, whether moW .effective programs were more or 

less cost|^. ' We also wanted this data in ord^r to assess the practicality of 

attempting to replidate particularly effective practices on any large scale. 
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Study Findin gs and nii-ir Implicationk 

. .■ In this discussion of the policy i^lications ot IDS findings 
«e.'re going to focus on two^of the variables considered above.'-individualized' 
■ , ■ -taction and inst^ctional setting. We're ^gnoftng cost and teacher 
traxnin^ because the findings in these areas tended to be non-findings. 
_ ■ mey are aapiy documented i„ the final study report produced by 
• • >=irschner et al ., In addition, upon further examination of these data „e ' ^ 
frankly find them to be generally rather non-thought^provoUng as well 
. _ ^= pap^r written for this p,nel by Hargot Kyitray ,nd Joy Frechtling does. ; 
however, contain some additional observations on cost analysis and what we 
^ reamed from a- slightly different- perspective;. 

.- ^ . Looking firs'At individualized instruction, let us repeat that L 

■ analyses to da^e have not revealed any practice or combination of practices ' 
to be. more effective than others. The most surprising filing was how 
well the CE students were doing across all our programs. While the ■ 
possibUity remains that additional .analyses will .yield ^ome more clear 
■•cut relationships, our findings in this area are consistent with a number' ^ 
of recent studies which show that lots of different instrucUonal approaches 
C . can work. Further, we suspect «iat-any relationships we might uncover;" • ' 
. «cause Of their level of complexity.' would probably be of more relevanae 
and interest to educators and educational researchers than to th'e polic,-' 
• Making audience. What then are the i^lications of our current findings 

from the policy standpoint? Are ,ur non-flings here more thought-provoking ' 
than those in other areas? * 

, . W. think the answer is "yes" and that our data in this'area can ' 
•be Of use in the policymaking process. Basically, the IDS findings r.infoul 
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the notion that instruction is a very complex process, which cannot now 

and may nevef^ be "reduced to some formula describing 'Vhat works." We" 

simply don*t know enough to require through law that cettain approaches 

be used. . , i . 

Further, the IDS data suggest that what might be called 
if - . / ^ 

^ "management-planning" variables may well play the critical role. Within 
reasonable limits, well planned and managed instructional programs using 
a variety of approaches may be successful. Of course, we are admittedly 
a Httle hazy on how to define good planning and management, although we' 
do have some starting points, and pronouncements in this area would also 
be premature. Nonetheless, there is enough evidence floating around to 
maJce a closer look at variables such as principal and teacher participation 
in planning, stability, and implementation levels of programs, and coordi- 
nation among teachers, a critical part of future examinations of what works. 

Turning to Instructional Setting, if is fair to say that our research 
has been useful and our comparisons of the achievement gains of students 
given mainstream instruction to those of students given pull-out 
instruction have in fact been able to inform policy debate. Last Fall, 
on the basis of analyses available, we indicated to Congress that pull-out 
instruction was not neces.sarily superior to mainstream. Our data showed 
mainstream instruction to be superior in the first grade for both 
reading and mathematics, pull-out instruction to be superior to third 
grade mathematics, and no differences in effectiveness fo third grade^ 

'reading. Now, as has been noteH, rechecking and cleaning of the data 
changes the picture slightly and the advantage of mainstream instruction 
appears even greater. Where differences are found (i.e. in first grade 
reading and math and third grade reading) gains in the mainstream setting 
were always signifilantly larger. • , 
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iVhat can we say from these 'data? Should* we amend our previous 
statement? to. Congress, taking an eve^V more aggressive stand in favor 
of the mainstream approach? Oui* answer is "no". Despite our revised 
1 findings we do not feel that av^^ing of the pendulum is called f errand < 
we would be uncomfortable concluding from the IDS data that districts 
should use only the mainstream approach. 

There are a number of reasons, for t-aking what might be labeled 
• a conservative stand*. . First, ieciuse the IDS findings are based on 

data from 'districts that- were "special", and not nationally representative 
especially with regard to using ma ins t reaming, we are hesitant to 
conclude that other districts jvould necessarily be as successful. 
Second, despite our data and other relevant information summarized 



recently by Dr. Glass, we have a gut feeling that mainstreaming may pose' 
problems for many teachers and simply may not be possible in some instances. 
We don't have any available "models^' for delivering CE instruction in 
this fashion nor is there sufficient * information on how to prepare 
teachers for what can be a very complex classroom management situation. 
It seems im^^ortant to make progress in thesQ areas before pushing for 
more widespread use of the approach. Finally," our , research needs ' 
replication and further analyses. The" XDS, despite all its virtues 
is only' one study, and ''convergent evidence from other work is not yet 
strong enough. If the IDS dkta is used in support of an open policy towards 
instructional setting, if it is used to stimulate further thinking on- 
how to provide mainstream instruction, fre think the study can be said 
to have served a very valuable policy function. 
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, There is one further issue which we have^.not focussed on yet"" 
but which bears mention. The IDS data^ seems to indicate that, at ■ 
, least m som6 classrooms in some district?. Title I is helping children 
' learn. We must b,e cautious about . the' conclusions we draw from, this as the 
IDS was not designed to provide an overall evaluation of how the .national 
Title I program ^is working. Nonetheless, in our sample classrooms the ' 
CE students were found to do well on a school year "basis and new data which - 
is just being analyzed suggests that i,mpressive gains exist .on a 12 month 
basis as well. The large summer drop-off effect reported in previous 
analyses does not seem to be replicated here. These findings represent. ) 
a hopeful change from the previous one's that have found little or no 
impact on achievement ^associated with Title l' pai^ticipation. 

We have done some speculation on why these findings differ from the others 
Clearly, we cannot rule our absolutely explanations such as sample bias and 
artifacts of the particular achievement test being used. However, it is 
just possible that a^^^ 10 years of practice sora^e programs in some places 
^e actually becoming more efi^ective, The policy implications of such 
a finding would indeed be tremendous, 



